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Abstract. This study aims at characterizing a reachable set of a hybrid dynamical system with 
a lag constraint in the switch control. The setting does not consider any controllability assumptions 
and uses a level-set approach. The approach consists in the introduction of on adequate hybrid 
optimal control problem with lag constraints on the switch control whose value function allows a 
characterization of the reachable set. The value function is in turn characterized by a system of 
quasi- variational inequalities (SQVI). We prove a comparison principle for the SQVI which shows 
uniqueness of its solution. A class of numerical finite differences schemes for solving the system of 
inequalities is proposed and the convergence of the numerical solution towards the value function 
is studied using the comparison principle. Some numerical examples illustrating the method are 
presented. Our study is motivated by an industrial application, namely, that of range extender 
electric vehicles. This class of electric vehicles uses an additional module - the range extender - as 
an extra source of energy in addition to its main source - a high voltage battery. The reachability 
study of this system is used to establish the maximum range of a simple vehicle model. 
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1. Introduction. This paper deals with the characterization of a reachable set 
of a hybrid dynamical system with a lag constraint in the switch control. The approach 
consists in the introduction of on adequate hybrid optimal control problem with lag 
constraints on the switch control whose value function allows a characterization of the 
reachable set. 

The term hybrid system refers to a general framework that can be used to model a 
large class of systems. Broadly speaking, they arise whenever a collection continuous- 
and discrete-time dynamics are put together in a single model. In that sense, the 
discrete dynamics may dictate switching between the continuous dynamics, jumps 
in the system trajectory or both. Moreover, they can contain specificities, as for 
instance, autonomous jumps and/or switches, time delay between discrete decisions, 
switching/jumping costs. This work considers a particular class of hybrid system 
where only switching between continuous dynamics are operated by the discrete logic, 
with no jumps in the trajectory, and there are no switching costs. In addition, switch 
decisions are constrained to be separated in time by a non-zero interval, fact which is 
referred to as switching lag. 

Before referring to the reachability problem in the hybrid setting, the main ideas 
are introduced in the non-hybrid framework. Given a time t > 0, a closed target set 
A"o and a closed admissible set K, considering a controlled dynamical system 

y(r) = f(T,y(T),u(r)), a.e. re[0,i], (1.1) 
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where / : ]R + x ]R d X U — > M d and it : TR + — > U is a measurable function, the 
reachable set Rx at time £ is defined as the set of all initial states x for which there 
exists a trajectory that stays inside K on [0, t] and arrives at the target: 



Rx (t) '■— i x I (y> u ) satisfies (1.1 1, with y(0) — x and y{t) £ X and y(s) £ K on [0,t]}. 



It is a known fact that the reachable set can be characterized by the the negative 
region of the value function of an optimal control problem. For this, following the 
idea introduced by Osher [18], one can consider the control problem defined by: 



v(x,t) := mf{va(y(t)) | (y,u) satisfies (1.1 1, with y(0) = x and y(s) € K on [0,t]}, 

(1.2) 

where vq is a Lipschitz continuous function satisfying vo(x) < x G Xq (for 
instance, Vq can be the signed distance dx to Xq). Under classical assumptions on 
the vector-field /, one can prove that the reachable set is given by 

R Xo (t) = {x € K,v(x,t) <0}. 

Moreover, when K is equal to IR d , the value function has been shown to be the unique 
viscosity solution of a Hamilton- Jacobi-Bellman (HJB) equation [2]: 

d t v + sup (-/(a, x, u) ■ V«) = on H d x (0,i], 

with the initial condition v(x, 0) = Wo(a;). 

When the set if is a subset of IR d (K ^ TR d ), the characterization of v by means of 
a HJB equation becomes a more delicate matter and usually requires some additional 
controllability properties (7j[l2]- However, it was pointed out in (7| that in case of 
state-constraints, the auxiliary control problem should be introduced as 



V(x,t) := inf < vo(y(t)) \/ max g(y(9)) I (y, u) satisfies (|1.1|), with y(0) = x 

1 v 9G[0,t] 1 ' 



(1.3) 

where g is any Lipschitz continuous function satisfying g(x) < <^=> x £ K (again, g 
can be the signed distance dx to K). This new control problem involves a supremum 
cost but does not include any state constraints. The reachability set is still given by 

R Xo {t) = {x e K,V(x,t) <0}, 

and the value function V is the unique viscosity solution of a variational inequality 

min(d t V+ sup(-/(a,x,«) • VV),V - g) =0 on (0,i] x IR d , 

with the initial condition v(x, 0) = min(uo(a;), g(x)), but no controllability assumption 
is needed. 

In this paper, we are interested in the extension of the reachability framework to 
some class of control problems of Hybrid systems. 

Let us recall that a hybrid dynamical system is a collection of controlled continuous- 
time processes selected through a high-level discrete control logic. A general frame- 
work for the (optimal) control hybrid dynamical systems was introduced in [8] . Several 
papers deal with the optimal control problem of hybrid systems, let us just mention 
here the papers of [TJ[l0j[l6j[20j where the optimality conditions in the form of Pon- 



tryagin' principle are studied and 11 14 22 where the HJB approach is analyzed 
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A feature of the hybrid system used in our work is a time lag between two consec- 
utive switching decisions. From the mathematical viewpoint this remove the partic- 
ularities linked to Zeno-like phenomena [2l|. Indeed, the collection of state spaces is 
divided in subsets labeled in three categories according to whether they characterize 
discrete decisions as optional, required (autonomous) or forbidden. Landing condi- 
tions ensure that whatever the region of the state space the state vector "lands" after 
a switch no other switch is possible by requiring a positive distance (in the Hausdorff 
sense) between the landing sets and the optional/autonomous switch sets. In the 
other hand, when allowed to switch freely without any costs, when no time interval 
is imposed between discrete transitions, a controller with a possibly infinite number 
of instantaneous switches may become admissible. Switch costs can be introduced in 
order to rule out this kind of strategy by the controller as it becomes over-expensive 
to switch to a particular mode using superfluous transitions. However, such costs do 
not make sense in the level-set approach used in this paper. 

Our study is motivated by an industrial application, namely, that of range exten- 
der electric vehicles. This class of electric vehicles uses an additional module - the 
range extender - as an extra source of energy in addition to its main energy source (a 
high voltage battery). For that matter, an adequate class of hybrid systems consid- 
ers only mode switching, i.e. switching between different continuous time dynamics, 
without any trajectory discontinuities. Moreover, no transition costs are taken into 
account and the switching decisions can be done freely without any penalty. Also mo- 
tivated by the application, the discrete control must respect a time interval of at least 
S > between two consecutive switching decisions. This decision lag condition can 



be viewed as replacing landing conditions and positive switch costs requirements 14 
Diffusion processes with impulse controls including switch lags are studied in [9 
where it is considered the idea of introducing a state variable to keep track of the 
time since the last discrete control decision. There, in addition, discrete decisions also 
suffer from a time delay before they can manifest in the continuous-time process. In 
that case, one has the possibility of scheduling discrete orders whenever the time for 
a decision to take place may be longer than that of deciding again. Then, the analysis 
also includes keeping record of the nature of this scheduled orders. This work inspired 
the idea of a state variable locking possible transitions used here. 

To study the reachability sets for our system, we follow the level set approach 
and adapt the ideas developed in |7| to hybrid systems by proposing a suitable control 
problem which allows us to handle in a convenient way the state constraints and the 
decision lag. It is proven that 

R x o(s) = {x | 3(q,p) e Q x P, v{x,q 1 p 1 s) < 0}. 

where v denotes the value function of a hybrid optimal control problem. Thus, through 
a characterization of v, one obtains the desired reachable set, defined in the hybrid 
context, x is in the (physical) state of the system, q is the discrete variable and 
p is a switch lock variable (defined further below). Here the main difficulty is to 
characterize the value function associated to the control problem. It turns out that 
this value function satisfies a quasi- variational HJB inequality system (in the viscosity 
sense) 

mm(d s u + d p u + H(s, x, q, V x u), u — ip(x)) = 0, (x, q, p, s) € Cl, (1-4) 
u(x, q,p, s) — (Mu)(x, q,p, s) — 0, p — (1-5) 
u{x, q,p, 0) = max(</>(x), (p(x)). (1-6) 
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where t > 0, d > 0, X = TR d , Q = {0,1}, P = (0,i], T = (0,t], fi = X X Q X 
P x T and where (j), ip : X — >• IR are target and obstacle indicator lunctions (defined 
properly further below). Here, M is a non-local switch operator that acts whenever 
the state variable p touches the boundary p = 0. Moreover, we give a comparison 
principle of this system. Usually, the proof of the comparison principle requires some 
transversality assumptions that do not make sense in the kind of applications we 
are interested in. However, while the decision lag complicates the structure of the 
problem it also plays a role in the proof of comparison principle (the same role that 
the transversality assumed in [l5j[22]) acting as a kind of landing condition in the 
sense explained above. The proof of the comparison principle is close to the one given 
in [3 14 and adapts the idea of using "friendly giants" -type functions. 



This work is motivated by an application in the automobile industry, namely, 
calculating the autonomy of a class of electric vehicles (EVs), the range extender 
electric vehicle (REEV) class, which posses two distinct sources of energy from which 
we can use to tract the vehicle. In this setting, the study aims at finding the control 
sequence of the two energy sources that allows the vehicle the reach the furthest 
possible point of a given route. The REEV is modeled as a hybrid dynamical system 
in which the state vector represents the energy capacities of the two different energy 
sources embedded in the vehicle. 

This paper is organized as follows: firstly, it describes the industrial application 
motivating the study and states the associated hybrid optimal control problem (in a 
slightly more general setting than that required for the application) . Then, the reach- 
able set and the value function are defined and a dynamic programming principle for 
the value function is obtained. The value function is shown to be Lipschitz continuous 
and also a solution of an system of quasi- variational inequalities (SQVI). It follows 
with the proof of a comparison principle for the SQVI that ensures uniqueness of 
its solution and is used to show the convergence of a class of numerical schemes for 
the computation of the value function. Lastly results of some numerical simulations 
evaluating the autonomy of a REEV toy model and illustrating the convergence of a 
discretization scheme are presented. 

2. Motivation and Problem Settings. 

2.1. Range Extender Electric Vehicles. A range extender (RE) electric ve- 
hicle is an electric vehicle that disposes of an additional source of energy besides 
the main high voltage (HV) traction battery. Both the vehicle's energy sources are 
considered to have normalized energetic capacities - thus valued between and 1. 

The controls available are the RE's state - on or off - and the power produced in 
the RE (and delivered into the powertrain). The power delivered into the powertrain 
is a non-negative piecewise continuous time function. The RE's state is controlled 
by a discrete sequence of switching orders decided and executed at discrete times. 
An important feature of the REEV model is a time interval 5 > imposed between 
two consecutive decisions times. From the physical viewpoint, this assumption incor- 
porates the fact that frequent switching of the RE is undesirable in order to avoid 
mechanical wear off and acoustic nuisance for the driver. 

The model considers that the vehicle's traction capability is conditioned to the 
existence of some electric energy in the battery. Since the vehicle must halt whenever 
there is no charge available in the battery, the objective of finding the vehicle auton- 
omy is summarized into finding the furthest point away from the vehicle geographic 
starting point where the battery is depleted for the first time. 
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2.2. Hybrid Dynamical System. Hybrid systems have some supervision logic 
that intervenes punctually between two or more continuous functions. The main 
elements of the class of hybrid dynamical systems considered in this work are a family 
of continuous dynamics (vector fields) / and continuous state spaces X, indexed by a 
discrete state q valued in a discrete set Q. Each continuous dynamic system f q valued 
in X q models a physical process controlled by a continuous control function u, from 
which the system is free to switch to another process f q > using a specified discrete 
control w and a discrete dynamics g. 

More precisely, the continuous state variable is denoted y and it is valued in the 
state space X — TR d . The discrete variable is q e Q = {0, 1, • • • , d q }, where d q is 
the number of possible dynamics that can operate the system, (for simplicity of the 
presentation, through this paper, we consider that X q = X for all q £ Q). Each 
of these dynamics models a different mode of operation of the system or a different 
physical process that the system is undergoing. Moreover, define a compact set K C X 
as the hybrid system admissible set, i.e., a set inside which the state must remain. 

The continuous control is supposed to be a measurable function u valued in a 
set that depends on the mode that is currently active U (q) . The discrete control is a 
sequence of switching decisions 

w = {(w 1 ,s 1 ),- ■ • , {wi,Si), (w t+1 ,s l+1 ), • • •}, (2.1) 

where each Si £ [0, oof and Wi G W(q) C {0, 1, • • • , d q }. The sequence of discrete 
switching decisions {wi}i>o (designating the new mode of operation) is associated 
with a sequence of switching times {si}i>o where each decision u>i is exerted at time 
Sj. The set of available discrete decisions, at time s, W(q(s)), depends on the discrete 
state variable and it corresponds to a decision of switching the system to another 
process Wi. 

The lag condition between switches is included by demanding that two switch 
orders must be separated by a time interval of 5 > 0, i.e., 

Si+i - Si> 5. 

Remark 2.1. Regarding the vehicle application, the vehicle's energy state is a 
two-dimensional vector i/£X = 1R 2 , where y — (j/1,1/2) denotes the state of charge 
of the battery and the fuel available in the range extender module. Each of these 
quantities are the image of the remaining energy in the battery and the RE respectively. 
It is clear here that the state variables have to be constrained to remain in the compact 
set K — [0, 1] 2 , where the energies quantities are normalized, q € Q — {0,1} is the 
RE state, indicating whether the RE is off (q = 0) or on (q = 1). The power output is 
a measurable function u(-) e U(q(-)) where U(-) is the admissible control set, compact 
subset of IR dependent naturally on the RE state. 

In this setting, given a discrete state q, the continuous control u steers the con- 
tinuous system 

j/(t) = f{T,y{r),u{T), qi ), for a.e. r e [0,t] (2.2) 

where some continuous dynamics f (•,•,•, Qi) is activated. / is a family of vector fields 
indexed by the discrete variable q. When q = qi, the corresponding vector field is 
active and dictates the evolution of the continuous state. At some isolated times 
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{sj}i>o, given by the discrete switching control sequence, the discrete dynamic g is 
activated 

qi-i= g(wi,qi) (2.3) 

and the continuous state follows another vector field /(•, •, •, qi—x). In the considered 
system, the discrete decisions only switch the continuous dynamics and introduce no 
discontinuities on the trajectory. In more general frameworks, we can also include 
jumps in the continuous state vector that can be used to model an instant change in 
the value of the state following a discrete decisions [5j[8]- 

Assume controlled continuous dynamics / and the discrete dynamic g satisfy the 
following: 

(HO) The continuous control is a measurable function u : [0, oo[— >• !R m such that 

u(t) € U(q(r)) for a.e. r £ [0,t]. 

(HI) There exists Lf > such that, for all s > 0, y, y' £ X, q £ Q and u £ U(q), 

\\f{s,y,u,q) - f(s,i/,u,q)\\ < L f \\y - y'\\, \\f(s,y,u,q)\\ < Lf. 

(H2) For all q £ Q, /(•, •, -,q) : [0,oo[xX x U — s- X is continuous and for all s £ 
[0, oof, x £ X,u £ U , f(s, x,u, ■) : Q — > X is continuous with respect to the 
discrete topology. 

(H3) For all s £ [0,t], y £ X and q £ Q, f(s, y, U, q) is a convex subset of X. 
(H4) There exists L g > such that, for all q £ Q and w £ W(q), 

\\9(w,q)\\ < L g 

(H5) g(-, ■) is continuous with respect to the discrete topology. 

Assumption (HI) ensures that a trajectory exists and that it is unique. Assump- 
tions (H2)-(H5) are used to prove the Lipschitz continuity of the value function and 
(H3) is needed in order to observe the compactness of the trajectory space. 

Denote A the space of hybrid controls a = (u, w). We precise the class of admis- 
sible controls A C A in the following definition: 

Definition 2.1. For a fixed t > a hybrid control a = (u,w) £ A is said to 
be admissible if the continuous control verifies (HO) and the discrete control sequence 
w = {u>i,Sj}i>o has increasing decision times 

Sl < s 2 < • • ■ < 8i < s t+1 <•••<*, (2.4) 

admissible decisions 

Vi > 0, Wi £ W(q( Sl )) c Q, (2.5) 

and verifies a decision lag 

Si+i - Si>6, (2.6) 

where 5 > 0. 

An important consequence in the definition of admissible control is the finitcness 
of the number of switch orders: 

Proposition 2.2. Fix s > 0. Let a £ A be an admissible hybrid control. Then, 
the discrete control sequence has at most N — [s/8\ switch decisions. 
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Fix t > 0. Given a hybrid control a £ A with N switch orders and given x € X, 
q € Q, the hybrid dynamical system is 



y( T ) = f( T ,y( T ), u ( T ),qi 

Qi-i= g(u>h<li), i = V 



re [o,t] 



y(t) = x 

qN = q 



(2.7) 
(2.8) 



Denote the solutions of ( 2.7 )-( 2.8 1 with final conditions x,q by y x . q -t and q x , q -,t- 
As pointed out, not all discrete control sequences are admissible. Only admissible 
control sequences engender admissible trajectories. Thus, given t > 0, x £ X and 
q € Q, the admissible trajectory set Yti'% is defined as 



r [o,t] 



- [o,t] 
a G ^4 and 



solution of ( 2.7 1-( 2.8 1 } 



(2.9) 



A consequence of proposition 2.2 and the above definition is the fmiteness of the 
number of discrete decisions in any admissible trajectory. Observe that the admissible 
trajectories set does not include the discrete trajectory. 



The hybrid control admissibility condition formulated as in conditions ( 2.4 )-( 2.6 1 



is not well adapted to a dynamic programming principle formulation, needed later 
on. In order to include the admissibility condition in the optimal control problem in 
a more suitable form, we introduce a new state variable it. Recall that the decision 
lag conditions implies that new switch orders are not available up to a time S since 
the last switch. The new variable is constructed such that at a given time r g [0,t], 
the value of 7r(r) measures the time since the last switch. The idea is to impose 
constraints on this new state variable and treat them more easily in the dynamic 
programming principle. Thus, if 7t(t) < 8 all switch decisions are blocked and if, 
conversely, tt(t) > 5 the system is free to switch. For that reason, this variable can 
be seen as a switch lock. 

Now given t > 0, r € [0, t] and a discrete control w = {wi, Si}i>o, the switch lock 
dynamics is defined by 



tt^t) = tt(t) 



5 + t if r < si 

inf Si < s r-Sj if r > s a 



(2.10) 



Indeed, once the discrete control is given, the trajectory tt(-) can be determined. 
Proceeding with the idea of adapting the admissibility condition in order to manip- 
ulate it in a dynamic programming principle, we wish to consider ir(t) = p, with 
p e P := (0,t], the final value of the switch lock variable trajectory and impose the 
lag condition under the form tt(s~) > 6 for all s$, where s~ denotes the limit to the 
left at the switching times Sj (notice that Tr(sf) — by construction). Then, since 
these conditions suffice to define an admissible discrete control set, while optimizing 
with respect to admissible functions, one needs only look within the set of hybrid 
controls that engenders a trajectory tt(-) with the appropriate structure. In other 
words, given t > 0, x £ X, q £ Q and p e P, define a admissible trajectory set S^' *j p 
as 



q !p . t solution of (|2.7[)-(|2.8[), 
tt(-) solution of KM, n(t)=p, n(sr)>8, i = l,---,N}. (2.11) 



The next lemma states a relation between sets Y and S: 



Lemma 2.3. Following the above definitions, sets (2.9) and (2.111 satisfy 

v x,q _ I I qx,q,p 
[0,t] ~ U [o.t] 
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Proof. The equivalence between Y^'% and {J pGP u [o t 



Stn1\ is obtained by construction. 



In the following of the paper, whenever we wish to call attention to the fact 



that the final conditions of (2.7), (2.8) and (2.10) are fixed, we denote their solutions 



respectively by y x ,q, P -t,qx,q, P -t,^x, q , P ,t- 

2.3. Reachability of Hybrid Dynamical Systems and Optimal Control 
Problem. Let Xq C X be the set of allowed initial states, i.e. the set of states from 



which the system ( 2.7 )-( 2.8 ) is allowed to start. Define the reachable set as the set 



of all points attainable by y after a time s starting within the set of allowed initial 
states X to be 

R x o{s) = {x\3q£Q, y x ^ s G Y™, y x , q -M € and y x , 9ia (0) G K, V0 G [0,s]} 

= {x | 3(q, P ) e Q x P, „„..,,,,. e , y x , q , P - s (0) e X° 

and y x , q , p;s (9) eK,yee [o, s]}. (2.12) 

In other words, the reachable set R x o(s) contains the values o{y Xyq . s (s), regardless 
of the final discrete state, for all admissible trajectories - i.e., trajectories obtained 
through an admissible hybrid control - starting within the set of possible initial states 
Xq, that never leave set K. 



Observe that (2.12) defines the reachable set Rxo(s) in terms of both admissible 
trajectory sets Y and S. 



Remark 2.2. In particular, the information contained in (2.12) allows one 



to determine the first time where the reachable set is empty. More precisely, given 
Xq C X , define s* > to be 



s* =mi{s | R x «(s) C 0}. 



(2.13) 



The time (2.13) is the autonomy of the hybrid system (2.7)-(2.8). Indeed, one can 
readily see that if no more admissible energy states are attainable after s* , any ad- 
missible trajectory must come to a stop beyond this time. Therefore, s* is the longest 
time during which the state remains inside K . 

The following proposition ensures that the space of admissible trajectories is a 
compact set. 

PROPOSITION 2.4. Given T' > 0, the admissible trajectory set Y, Q is a compact 

set in C([0, T']) endowed with the topology W ' . 

Proof. Fix q £ Q and < s < t < T' . Consider a bounded admissible continuous 
control sequence u n G L 1 ([s,t]). Since u n is bounded, there exists a subsequence u n . 
such that u n . — 1 u in L 1 ([s,i]). Invoking (HI) — (H2), we have y u ™ = y n y in 
1 / F 1,1 ([s,t]). Since W" 1,:L ([s, t]) is compactly embedded in C°([0, f]), we get the strong 
convergence of the solution y n — > y in C°([s, t]). Hypothesis (H3) guarantees that the 
limit function y is a solution of (2.7). Because all controls u n and the limit control u 



are admissible, y is an admissible solution. 

So far, the proof shows that the limit trajectory is admissible when q is hold 
constant. Consider a sequence of admissible discrete control sequences (w) n where 
the number of switching orders, < k n < \T /5\ may depend on n. Since each term 
of this sequence has a (first) discrete component and is bounded on the (second) 
continuous component, then, as n — ¥ oo there exists a subsequence {w) ni and A > 



such that q n 



q for all I > A. This implies k n — > k. As the number of switches 
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is constant from the A th term and s, t are arbitrary, one can obtain, using the limit 
discrete control sequence w, the time intervals [si_i, Sj] over which is constant and 
the argument in the first paragraph of the proof. Because the trajectory is continuous 
and admissible on all time intervals [s,_i, Sj], i = 1, • • • , if, it is admissible on [0, T]. 

Moreover, observe that Vs £ [0, t], y n {s) £ K. Since for all s £ [0, t], y n {s) — > y(s), 
by the compactness of K we get that Vs £ [0, t], y(s) £ K, which completes the proof. 
□ 

Remark 2.3. The arguments presented in the above proof can be slightly modified 
to show that the admissible trajectory set with fixed final p, S^ q ^ is compact. Also 
in a similar way, the proof can be adapted to show that the reachable Rx° is closed. 
Indeed, by the compactness of set Xq, a sequence of initial conditions (yo)n G Xq, 
associated with admissible trajectories y n £ Y, '^,,,, converges to yo £ Xq which is also 
the initial condition for the limiting trajectory y n — » y. 

In order to characterize the reachable set Rx this paper follows the classic level- 
set approach IT8J . The idea is to describe (2.12) as the negative region of a function 
v. It is well known that the function v can be defined as the value function of some 
optimal control problem. In the case of system (2.7)-(2.8 1, v happens to be the value 
function of a hybrid optimal control problem. 

Consider a Lipschitz continuous function (f> : X IR such that 

<t>(x) <0<^xeX°. 

Such a function always exists - for instance, the signed distance function dx from 
the set X . For Lk > 0, one can construct a bounded function <f> : X — > ]R as 



4>{x) — max(min(0(a;), L 



K) 



-L 



K) 



(2.14) 



For a given point s > and hybrid state vector (x,q,p) £ X x Q x P, define the 
value function to be 



v (x,q,p,s) = ml {(f>(y Xtq<p . >s (0)) \ y x . q . P]S {6) € K, V6 1 e [0,s]} 



(2.15) 



Observe that (2.12| works as a level-set to the negative part of (2.15). Indeed, since 



(2.15) contains only admissible trajectories that remain in K, by (2.14) implies that 
vq(x, q,p, s) is negative if and only if y x ,q,p; S (0) is inside Xq, which in turn implies 
that x € Rx° (s). 



Remark however that when defining the value function with (2.15), one includes 
state constraints, with the condition that y x ,q,p-s(0) € K for all times. When K ^ TR d , 
one cannot expect v to be continuous and the HJ equation associated with (2.15) may 
have several solutions. In order to bypass such regularity issues, this paper follows 
the idea of (6j[7|. Define a Lipschitz continuous function (p : X — > ]R to be 



(p{x) < <^> x e K , 



and 



ip{x) 



c(mm(tp(x),L K ), -L K ). 



(2.16) 



Then, for a given s > and (x,q,p) £ X x Q x P, define a total penalization function 
to be 



J(x,q,P,s;y) = (j>(y. 



9e[o,s\ 
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and then, the optimal value 



v(x,q,p,s) = inf J(x,q,p,s;y). 



(2.17) 



Observe that (2.15) and (2.17) are bounded thanks to the constructions (2.14) and 



( 2.16 ) respectively. The idea in place is that one needs only to look at the sign of vq or 



v to obtain information about the reachable set. Therefore, the bound Lk removes the 
necessity of dealing with an unbounded value function besides providing a convenient 



value for numerical computations. In order to ensure that constructions (2.14) and 



(2.16) do not interfere with the original problem's formulation (in the sense that using 
4>, if or cj), (p should yield the same results), given s, Xq and x, assuming one does use 
signed distance functions to sets X , K, it suffices to take Lk > sup^g^ x'e Lf \ x ~ x I s , 
where Lf is the Lipschitz constant of /. 



3. Main Results. The next proposition certifies that ( 2.12 ) is indeed a level-set 



of (2.151 and (2.17) 



Proposition 3.1. Assume (Hl)-(HS). Define Lipschitz continuous functions (f> 
and if by (2.14) and (2.16) respectively. Define value functions Vq and v by (2.15) 



and (2.17) respectively. Then, for s > 0, the reachable set is given by 



R x o(s) = {x | 3(q,p) G QxP, v {x,q,p,s) < 0} = {x | B(q,p) G QxP, v(x,q,p,s) < 0} 

(3.1) 

Proof. The proof begins by showing that vq(x, q,p, s) < => v(x,q,p, s) < 0. 
Assume vq(x, q, p, s) < 0. Then, using lemma 2.4 there exists an admissible trajectory 
such that 

0(tf»,«^»(O)) < o, yx, q , P ;s(9) e K, G [0,S], 

Thus, maxo £ [o, s ] f(Vx,g,p;s(&)) < and 

v(x,q,p, s) < max(<p(y Xl q tP . s (0)), max (p{y x . q , p - s {9))) < 

9e[o,s] 

Now, show that v{x,q,p, s) < =>■ Vo(x,q,p, s) < 0. Assume v(x,q,p, s) < 0. 



Then, by lemma 2.4 there exists a trajectory that verifies 



ma,x(4>(y XtqtP . s (0)), max (p(y x , q , p;s (d))) < 0. 
9e[o,s] 

By the definition of <p, \/9 G [0, s], 

max(ip(y Xt q tP ; S (6))) < =^> y x , q ^- s (d) G K, 

which implies vo(x,q,p,s) < 0. Therefore, u and v have the same negative regions. 

Now, assume y x ,q,p;s{ s ) & Rx°( s )- Then, by definition, there exists (q,p) G QxP 
and an admissible trajectory such that y x ,q,p-s(9) G K for all time and y x ^ jP - s (0) G Xo. 
This implies that maxg e [ ,s] ( l f(y x , q , P ;s{0))) < and <j){y x ,q,p;s(0)) < 0. It follows that 
v(x,q,p,s) < J(x,q,p,s;y) <0. 

Conversely, assume v(x, q,p, s) < 0. For any optimal trajectory y (which is admis- 
sible thanks to proposition 2.4) v(x,q,p,s) — J(x,q,p,s;y) < 0. Since the maximum 



of the two quantities is non positive only if they are both non positive one can draw 
the desired conclusion. □ 
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Proposition (3.1| sets the equivalence between (2.12) and the negative regions of 



(2.15) and (2.17). In particular, it states that it suffices to computes v or vq in order 



to obtain information about Rx - I n this sense, this paper focuses on (2.17), which 
is associated with an optimal control problem with no state constraints. 



In the sequel, it is shown that (2.17) is the unique (viscosity) solution of a quasi- 



variational inequalities' system. The hrst step is to state a dynamic programming 



principle for (2.17) 



First, we present some preliminary notation. Given t > 0, set T = (0, t], il 
X x Q x P x T and denote its closure by f2. For a fixed po G P, define Sl\ Po 
X x Q x {p } x T and denote Q\ Po the closure of fi| po . Define 



V(fi) := {v | v : Q -> R, v bounded }, 
V(f2| Po ) := {v | for p e P, v. Q\ Po IR, v bounded } 



(3.2) 
(3.3) 



For v G V(f2), denote its upper and lower envelope at point (x, q,p, s) G f2 respectively 
as and u*: 



v*(x,q,p,s) = \]mswpv(x n ,q n ,p n ,s n ) 

s n —±s 

v*(x,q,p,s) = hmmf v(x ni q n ,p n , s n ) 

s« — > s 



(3.4) 
(3.5) 



In the case where po G P is fixed and w G V(f2| Po ), the upper and lower envelopes of 
v are also given by (3.4), (3.5 ) with p n = p for all n. 



Now, fix p — and define the non-local switch operators M, M + ,M : V(f2|o) 
V(H| ) to be 



(Mv)(x,q,0,s) = inf v(x,g(w,q),p\s) 

wEW(q) 
p'>8 

(M + v)(x,q,0,s) = inf v* (x,g(w,q),p' , s) 

wEW(q) 
p'>8 

(M~v)(x,q,0,s) = inf v*(x, g(w, q),p', s) 

weW(q) 
p'>8 



The action of these operators on the value function represents a switch that 
respects the lag constraint. They operate whenever a switch is activated, which is 
equivalent to the condition p — 0. Therefore, they are defined only for a fixed p = 0. 
Let us recall here some classical properties of operators M, M + and M~ (adapted 
from [22]): 

Lemma 3.2. Let v G BV(U). ThenM+v* G BUSC(U) andM-v* e BLSC(ty. 
Moreover (Mv)* < M+v* and (Ms), > M~v*. 

Proof. Fix q G Q, p — and e > 0. Let w* G IF(a;, q) and p* > be such that for 
all x G X and s G T, (M + v*)(x, q,p, s) > v*(x, g(w* , q),p* , s) — e. Consider sequences 
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X n r X and s n — > s. Then 

M + v*(x,q,0,s) > v*(x,g(w*,q),p*,s) ~ e 

> limsupu* (x n , g(w* ,q),p* , s„) - e 

x n — + x 

>limsup inf v*(x n , g(w, q),p', s n ) - e 

x n — >x weW(x n ,q),p'>S 

= linisup(M + ?;*)(x„, q, 0, s n ) - e. 

x n ^x 
s n —ts 

Notice that q and p are held constant throughout the inequalities and thus, the limsup 
of the jump operator considering only sequences x n — > x and s n — > s corresponds to 
its envelope at the limit point. Then, by the arbitrariness of e, this proves the upper 
semi-continuity of M + v* . The lower semi-continuity of M~v* can be obtained in a 
similar fashion. 

Now, observe that Mv < M + v* . Taking the upper envelope of each side, one 
obtains: 

(Mv)* < (M+v*)* = M+v*. 
By the same kind of reasoning Mv > M~~v* and 

(Mil), > (M~v*), = M~v*. 

□ 



The next proposition is the dynamic programming principle verified by (2.17): 
Proposition 3.3. The value function (2.171 satisfies the following dynamic 
programming principle: 
(i) For s=0, 

v(x, q,p, 0) = m&x((f>(x), <p(x)), V(x,q,p) € X x Q x P, (3.6) 

(ii) For p = , 

v(x,q,0,s) = (Mv)(x,q,0,s), (x,q, s) € X X Q X T, (3.7) 

(Hi) For (x,q,p,s) £ define the non-intervention zone as S = (0,pAs). Then, 
for h € E, 

v(x,q,p,s)= inf \ v(y x ^ p . h (s - h), q,p - h, s - h) \J max if(y x , q , p -h(0)) 
s {s-h 3 ] I ee[s-h,s] 

(3-8) 

Proof. The dynamic programming principle is composed of three parts. 

(i) : Equality ( |3.6[ ) is obtained directly by definition (2.17). 

(ii) : " < ". Let (x,q,s) G X x Q x T and p = 0. Consider a hybrid control 
a = (u(-), {u)i, Si}fLi) and an associated trajectory y a . Construct a control a = 
(u(-), {Wi, Sijili 1 ) w ith associated trajectory y a , where u = u, Wi = Wi and s$ = s* 
for z = 1, • • • , — 1 and sn = s, wn = w' . Then, one obtains, 

v(x,q,0,s) < J(x,q,0,s;y a ) 

= J(x,g(w',q),p',s;y a ), 
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where the controller must respect the condition p' > 5 for it to be admissible. Since 
a is arbitrary, one can choose it such that 



v(x,q,0,s) < 



y-es 



inf 

(x,g(w r ,q),p') 
[0,3] 



J(x,g(w' ,q),p' \s;y a ) + e% 



= v(x,g(w',q),p',s) +e 1; 
where ei > 0. Now, choose the last switch w' and p' such that 
v(x,q,0,s)< inf v(x,g(w',q),p',s) + ei 

w'eW(q) 
p>8 

= (Mv)(x,q,p,s) + e 1 . 

" > ". For (x,q,p) e X x Q x P and p = 0, there always exists an admissible 
control that a c , such that there exists ti > and, 

v(x,q,0,s) + e 2 > J(x, q, 0, s; y a <) 

Using the same hybrid control constructions as in the "<" case, one obtains 

J(x,q,0,s;y°") = J(x, g{w' ,q),p' , s;y ac ) 

> v(x,g(w',q),p',s) 

> inf v(x,g(w' \q),p / ,s) 

w'eW(q) 
p'>5 

= {Mv)(x,q,0,s). 



Relation (3.7) is obtained by the arbitrariness of both e±, ti- 



(hi): " < ". For (x,q,p, s) G Q and < h < p A s, ( |2.17| yields 
v(x,q,p, s) < max ( I (f>{yx, q , P -s{0))\/ max f(y x , q , p;s (9)) ) , max Lp(y x ^ p . s (0)) 

\\ 8E[0,s—h] I fle[s — h.s] 

(3.9) 

for any y G S^ 9 f. By the choice of h, there is no switching between times s — h and 
s. Write the admissible control a = (u,w) as ao = (uq,w) and a\ — (ui,w) with 

uq(s) — u(s), s G [0, s — h], 
ui(s) — u(s), s G (s — h, s}. 



Since a is admissible, both controls ao,a\ are also admissible. Denote the trajectory 



associated with controls a, oo,ai respectively by y a , y°, y 1 . Then, y a G Sfi: q f and by 



continuity of the trajectory we achieve the following decomposition: 

c qV 1 (s-h),q,p-h 



1 ca;,g,p c cy 

y t or s _ hs] , y g ^> [0j 



[0,s-h] 



The above decomposition together with inequality (3.9) yields 

v(x,q,p,s) <max( U(j/°(0))\/ max^iptftf)) J , _max ^(y 1 ^)) ) , 



ee[o,s-/i] 



ee[s-fe,s] 
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And one concludes after minimizing with respect to the trajectories associated 
with do and a\. 

The " > " part uses a particular e-optimal controller and the same decomposition, 
allowing to conclude by the arbitrariness of e. This is possible because there is no 
switching between s — h and s. 
□ 

A direct consequence of proposition |3.3| is the Lipschitz continuity of the value 
function, stated in the next proposition: 

Proposition 3.4. Assume (H1)-(H2). Define Lipschitz continuous functions <p 



and if by (2.14) and (2.16), with Lipschitz constants L^ and L v respectively. Then, 
for p > 0, (2.17) is Lipschitz continuous. 

Proof. Fix s.s' > 0, x, x' <G X, q G Q, p > 0. Then, using max(A, B) - 
max(C, D) < max(^4 — B,C — D), one obtains, 

\v(x,q,p,s) -V(x',q,p,s)\ < max (\<t>(y x ,q,p;a(0)) ~ <P(Vx' ,q,p;s(0))\ , 

max \<p(y x , q>p . ia (6)) - <p(y x >, q ,p; S (0))\ 
< max \y x>q>p . s (0) - y x > , q , p - a (0)\ , 

L <p (\yx,q,p;s(0) - y X ',q,p;s(0)\) 

oe[o,s] 

<L v \x — x'\, 

where L v = max(i0, L ip )e L,s . 

Now, take h > and observe that v(x, q,p, s) > <p(x). Then, 



\v(x, q,p + h, s + h) — v(x, q,p, s)\ < max 



v(y x ,q, P+ h; S +h(s),q,p,s) - v{x,q,p,s) 



max ip{y x ,q,p; S+h (0)) - <p(x) 
ee[s,s+h] 



< max L 



yx,q,,p+h;s+h{ s ) y X ,q,p;s(s) 

L v max \y s ,q,p+h;s+h(Q) — y X ,q,p;s(s)\ ) 
9€[s,s+h] J 

< Lf max(L t) , L v )h. 



□ 

In order to proceed to the HJB equations, define the Hamiltionian to be 

H(s,x,q,z)= sup f(s,x,u,q)-z (3.10) 

ueU(q) 

Before stating the next result, we recall the notion of viscosity solution [13] used 
throughout this paper. 

Definition 3.5. A function ui (resp. U2) upper semi- continuous (u.s.c.) (resp. 
lower semi- continuous (l.s.c) is a viscosity subsolution (resp. super solution) if there 
exists a continuously differ entiable function ij) such that U\ — tj) has a local maximum 
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(resp. U2 — ip has a local minimum) at (x, q,p, s) G fi and 

ip)/\u 1 -<p(x)<Oif(x,q,p,s)en (3.11) 

Ui(x,q,p,s) < (M + Ui)(x,q,p,s) ifp = 0, (3-12) 

ui(x,q,p,s) < max((f>(x) 1 ip(x)) if s = 0, (3.13) 

(with the inequalities signs inversed and M~ instead of M + for ui). A bounded 
function u is a (viscosity) solution of (3.14)-(3.16) if u* is a subsolution and is a 
super solution. 



The next statement shows that the value function defined in (2.17) is a solution 
of a quasi-variational system. 

Theorem 3.6. Assume (H1)-(H5). Let the Lipschitz functions <f> and if be 



defined by (2.14) and (2.16) respectively. Then, the Lipschitz, bounded value function 



v defined in (2.17) is a viscosity solution of the quasi-variational inequality 



d s v + d p v + H(s,x,q, V x w) f\v - tp(x) = 0,V(x,q,p,s) E ft, (3.14) 
v(x,q,0,s) = (Mv)(x,q,0,s),V(x,q,s) eXxQx[0,co[ (3.15) 
v(x,q,p,Q) = msx.(4>(x),(p{x)),\/(x,q,p) E X x Q x P. (3.16) 



Proof. By definition, v satisfies the initial condition (3.16). The boundary condi- 
tion (3.15) is deducted from proposition 3.3 Now, we proceed to show that (i) v is a 
supersolution and (ii) a subsolution of (3.14): 

First, let us prove the supersolution property (i). To satisfy min(A, B) > one 
needs to show A > and B > 0. Since v — (fi(x) > 0, it is immediate that B > 0. 
Now, consider < h < p A s. Let ip be a continuously differentiable function such 
that v — ip attains a minimum at (x,q,p, s). Then, using proposition (3.3)(m) and 
selecting an e-optimal controller, dependent on h, with associated trajectory y e x q p . s , 
it follows that 



tp(x, q, p, s) = v(x, q, p, s) > inf v{y XA . p . s (s 



h), q,p — h, s — h) 



and then, 



> v (y e x,q, P ;s( s -h),q,p-h,s-h)- 
= ^(yl, q ,p;s( s - h),q,p- h,s-h) 

ip(x, q,p, s) - ip(y £ X:q:P . s {s - h), q,p- h,s- h)> -he. 



he 
-he 



Since the control domain is bounded and using the continuity of /, p and ip we divide 
by h and take the limit h — > to obtain 

d s ip + d p ip + H(s, x, q, V x ip) > -e 

and conclude that A > by the arbitrariness of e. 

For (ii), observe that for min(A, B) < it suffices to show that A < or B < 0. 
If v(x,q,p, s) = ip(x), it implies B < 0. On the contrary, if v[x, q,p, s) > f(x), then 
there exists a S 9 h > small enough so that 

v{yx, q , P ;s(s -h),q,p-h,s-h)> max ip(y^ q>p . s (9)) 
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strictly, using the Lipschitz continuity of /, p and the compactness of U (which ensures 
the trajectories will remain near each other). Thus, proposition |3.3[m) yields 



v(x,q,p,s) = inf v(y% (s - h),q,p - h, s - h). 



Fix an arbitrary u G U and consider a constant control u(s) = u for < s < h. 
Let tp be a continuously diffcrentiable function such that v — ip attains a maximum 
at (x,q,p,s). Also, without loss of generality, assume that v(x,q,p,s) — ip(x,q,p,s). 
Hence, 

v(x, q,p, s) < v(yl qp . s (s - h), q,p- h,s- h) 
< ^(yx, q ,p- s ( s - h),q,p- h,s- h) 

and by dividing by h and taking h — > one obtains 

<9 S V> + d p ip + f(s, x, u) ■ V x ip < 0. 

Since u is arbitrary and admissible, we conclude that A < 0, which completes the 
proof. □ 

Theorem |3.6| provides a convenient way to characterize the value function whose 



level-set is the reachable set defined in (2.12 1. However, in order to be sure that the 



solution that stems from (3.14)-(3.16) corresponds to (2.17), a uniqueness result is 
necessary. This is achieved by a comparison principle which is stated in the next 
theorem. Let BUSC(fl) and BLSC(VL) respectively be the space of u.s.c. and l.s.c. 
functions defined over the set f2. 

Theorem 3.7. Let u\ e BUSC(Cl) and u 2 G BLSC(ft) be, respectively, sub- 
and supersolution of 

d s u + d p u + H(s, x, q, V x u) Aw - <p(x)) — 0, V(x, q,p, s) G O, (3-17) 
u(x,q,0,s) - (Mu)(x,q,Q,s) = 0,V(x,q,s) G X x Q x [0,oo[(3.18) 
u(x, q,p, 0) = max(<f>(x),(p(x)),W(x, q,p) G X x Q x P. (3.19) 

Then, u\ < u 2 in Q. 

The proof is inspired by earlier work on uniqueness results for hybrid control 
problems. The idea is to show that u\ < u 2 in all domain O and then on the bound- 
ary p = 0. The main difficulty arises when dealing with points in the boundary p = 
where the system has a switching condition given by a non-local switch operator. This 
is tackled by the utilization of "friendly giant" -like test functions [3], [17]. Classically, 
these functions are used to prove uniqueness for elliptic problems with unbounded 
value functions where they serve to localize some arguments regardless of the func- 
tion's possible growth at infinity. This feature proves itself very useful in our case 
because one can properly split the domain in no-switching and switching regions. In 
this work, the lag condition for the switch serves as an equivalent to the "landing 
condition"- which states that after an autonomous switch the system must land at 
some positive distance away from the autonomous switch set [14] , [22] . 

Proof. Let fl be defined as above, dfl\x = X x Q x P x {0} and dfl\p = X xQx 
{0}xT. 

First, the comparison principle is proved for 8Q\t (case 1), followed by Q (case 
2) and finally for dil\p (case 3), which concludes the proof for fi. 
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Case 1: At a point {x, q,p, i) <E <9f2|r, from the sub- and supersolution properties, 

ui(x, q,p, 0) — max((j)(x), <p(x)) < 0, 
—u 2 (x, q,p, 0) + max(4>(x), tp(x)) < 0, 

which readily yields u\ < u 2 in 3Q\t- 

Case 2: Start by using to sub- and supersolution properties of U\,U2 to obtain, 
in i7, 



min(i9 s Mi + d p ui + H(s, x, q, V x ui),ui - <p(x)) < 0, 
min(d s u 2 + d p u 2 + H(s, x, q, V I u 2 ),M2 - fix)) > 0. 



Expression (3.21) implies that both 



and 



u 2 > <p(x) 



(3.20) 
(3.21) 



(3.22) 



d s u 2 + d p u 2 + H(s, x, q, V x u 2 ) > 0. 



(3.23) 



From (3.201, one has to consider two possibilities. The first one is when U\ < <p(x). 
If so, together with (3.22), one has immediately u% < u 2 . Now, if d s ui + d p u\ + 
H(s,x,q,V x Ui) < 0, one turns to (3.23). 

Define v = m — u 2 . Notice that v £ BUSC(Q). The next step is to show that v 
is a subsolution of 



d s v + d p v + H(s, x, q, V x v) = 



(3.24) 



at (x,q,p,s). 

Let V € C 2 (£l), bounded, be such that v — tp has a strict local maximum at 
(x, q,p, s) € f2. Define auxiliary functions over f2* x ft 1 , i = 0, 1 as 

^l{x,p,s,£„TT,c;) = u x {x,i,p,s) - u 2 (£,i,7r,<?) - ip{x,p,s) (3.25) 

2e 2e 2e 

Because the boundedness of tp, u\ and u 2 the suprema points are finite, for each 
i = 0, 1. Denote (a; e ,p e , s e , £ e , 7r e , <r e ) G x f2' a point such that 

$«(ai e ,p e ,S e ,^e,7re,?e) = SUp $*(x,p,S,C,7T,?). 

f29xf25 

The following lemma establishes some estimations needed further in the proof: 
Lemma 3.8. Define <£>* and (x e ,p e , s e , £ e , 7r e , <; e ) as above. Then, as e — > 0, 



K-g £ | 2 ;Q b<-^| 2 



0, 



0. 



tee 
|z e - £e| 0, |p £ - 7r e | -s- 0, |s £ -q|->0, 



and (x e ,p c ,s £ ,£ e ,7r e ,? c ) ->• (x,p, s, s) 
Proof. Writing 

2®l(x e ,p e ,s e ,£ e ,>K e ,<; e ) > $l(x e ,p e ,s e ,x e ,p e ,s e ) + $ l e (^,7r e ,Q,^,7r e ,Q), 
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for i = 0,1, one obtains, 

l*-£l 2 , \p-A 2 , \s- 



1p(x e ,p e ,S e ) - VKCcToSe), 



which means that, since if), u\ and u<i are bounded that 



< c*, 



IP. 



< c*, 



< c* 



(3.26) 



where C* depends on the sup \u± |, sup|ii2|, sup |^| and is independent of e. Expression 
(137261) yields 



•Sel < VeC* 



which implies that the doubled terms tend to zero. 

Since (x,q,p,s) is a strict maximum of v — ip, one gets (x e ,p e , s e , £ e , ir e , — > 
(x,p,s,X,p,s). Remark that, since p > 0, one can always choose a suitable subse- 
quence e„ — ¥ such that all p £n > 0, avoiding thus touching the switching boundary. 
□ 

A straightforward calculation allows to show that there exists a, b £ IR such that 

(a,b,D e ) e L> - u 2 (£ e ,g,7r e ,^) 
(a + d s %l), b + d p ip, D e + V X V) S L> + ui(xe, g,p e , s e ), 



where D , D + respectively denote the sub- and super differential 13 and D e — 
2\x e — £ e |/e, which implies 

a + b + H(^,^,q,D e ) > 
a + d s i/j + b + d p il> + H(s e ,x e ,q,D e +V x il>) < 0, 

which in turn yields, as e — > 0, 

d s iJj + d p Tp-L f \V x ip\ < 

at (x,q,p,s) £ il. By adequately choosing the test functions ip, one can repeat the 
arguments to show that this assertion holds for any point in f2. Thus, this establishes 
that v is a subsolution of (3.24) in f2. 

Now, take k > and define a non-decreasing differentiable function \k '■ (— oo, 0) — > 
]R + such that 

X«(x) =0, x < -k ; Xk(ie) -> oo, x -> 0. 

Take ry > 0,and define a test function 

v(x,p,s) = 77s 2 + Xk(-p)- 

Observe that v — v achieves a maximum at a finite point (xo, q, po, sq) £ fl. Since 
k can be made arbitrarily small one can consider po > k without loss of generality. 
Therefore, using the subsolution property of v, by a straightforward calculation one 
has 

2r]s < 0, 
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since x'ni~ po) = 0. The above inequality implies that so = 0. Noticing that 
v(.x ,p ,s ) = v(x ,q,p ,s ) = 0, it follows 

v(x,q,p,s) < i]s 2 +Xn(-p) 

for all s £ T, x £ X and p > k. Letting r\ — s- 0, k — > and from the arbitrariness of 
q, we conclude that v < in f2. 

Case 3: In this case the switch lock variable arrives at the boundary of the 
domain, incurring thus a switch, as all others variables remain inside the domain. 
For all (xo,qo,po, Sq) € dQ\p, for any p > 5 one has (using case 2 and noticing that 
M + ui = Mui and M~u 2 = Mu 2 ) 

(M + ui)(x ,q ,po,s ) < ui{x a ,q Q ,p, s ) < u 2 (x , q ,p, s Q ). 

Taking the infimum with respect to p, the above expression yields M + u\ < M~u 2 in 
dfl\p. This suffices to conclude, since that by the sub- and supersolution properties 

v = ui — u 2 < M + u\ — M~u 2 . 

□ 

4. Numerical Analysis. 

4.1. Numerical Scheme and Convergence. Equations ( |3.14 |-(3.16l can be 

solved using a finite differences scheme. This section proposes a class of discretization 
schemes and shows its convergence using the Barles-Souganidis [4] framework. 

Set mesh sizes Ax > 0, Ap > 0, At > and denote the discrete grid point by 
(xj,pk, s n ), where xi = I Ax, pk = kAp and s n = nAt, with / £ 7L d and k, n integers. 
The approximation of the value function is denoted 

v{x I ,q,p k ,s n ) = q v^ k 

and the penalization functions arc denoted 4>(xj) = <pi, ip(xi) = ifi- Define the 
following grids: 

G* = I Ax xQx Ap{0, 1, • • • , n p } x At{0, 1, ■ • • , n s }, 
G% = At{0, 1, • • • , n s } x I Ax x Q 

and the discrete space gradient at point xi for any general function /i: 

D ± ^{X I ) = D ± m = (D±»i, • • • , D± dt ii) , 

where 

r>± , M/i-± - Mi 

D x^i = ± t , 

J Ax 

with 

I h± = (h, ■ ■ ■ ± ,id). 



Define a numerical Hamiltonian % : 67^ x Wl d x IR d — > IR destined to be an 
approximation of H. We assume that T-L verifies the following hypothesis: 
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(H6) There exists Lh x , Lh 2 > such that, for all s, x, q E and A + ,A , B + ,B 6 

\n{s,x,q,A + ,A-)-U{s,x,q,B + ^B-)\ < L Hl {\\A+ - B+\\ + \\A~ - B~\\ 
\\H(s,x, q ,A + ,A-)\\<L H2 (\\A+ + A-\\). 



(H7) The Hamiltonian satisfies the monotonicity condition for all s,x,q £ and 
almost every A + ,A~ e lR d : 

d A+ H(s,x,q,A + ,A-) < 0, and d A -H(s, x, q, A + , A~) > 0. 



(H8) There exists L# 3 > such that for all s, x, q € G*, s', x' ,q' el x X x Q and 
Ae ]R d , 

\H(s, x, q, A, A) - H(s', x', q', A)\ < L Ha (\s - s'\ + \ \x - x'\ \ + \q - q'\). 



Let $ : Q M, h = (Ax, Ap, At) and set 

Sh (x, q,P, s, A; $) = min (A - <pi,H(s, x, q, D + $(x, q,p, s),D~$(x, q,p, s)) + 

A - <f>(x,g,p,s) A-$(x,g,p- Ap,s + At) 
At Ap 

Now, consider the following scheme 

„, . f S%(x,q,p,s,\;$) if (x,q,p,s)eQ 

S h (x,q,p,s,X;<S?) = < 

A - xsmx weW{Xtq)tP ,> s ${x,g{w,q),p ,s) it p = 0, 

(4-1) 

along with the following operator 



J-(x, q,p, s, u, Vu) 



u — ip(x) f\d s u + d p u + H(s,x,q,V x ii) if (x, q,p, s) € Q 
u(x,q,p,s) — (Mu)(x,q,p,s) if p = 0. 

(4.2) 

PROPOSITION 4.1. Let $ e C£°(f2). f/nder hypothesis (HQ - H8) and the CFL 
condition 



J2d At n + d A7 n) <i 

i=l / 



the discretization scheme (4.1) of (4.2) is stable, monotone and consistent. 

Moreover, a solution Uh of (4.1) converges towards the solution u of ( |4.2[ ) as 
h ->■ 0. 

Proof. The proof follows the lines used in the framework of Barles-Souganidis [i]. 
The goal is to show that the numerical scheme solutions' envelopes 

u(x' \q' ,p' ', s') = liminf Uh(x,q,p,s) 

(x,q,p,s)—>(x',q\p',s') 
/l-S-0 

u(x' , s' ,p' , s') = limsup uu{x,q,p,s), 

(x,q,p,s) — > (x ,q ,p ,s ) 
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are respectively supersolution and subsolution ol (4.2 1. Then, using the comparison 



principle in theorem |3.7[ one obtains u < u. However, since the inverse inequality is 
immediate (using the definition of limsup and liminf, one gets u = u = u achieving 
thus the convergence. 

Only the subsolution property of u is presented next, the proof of the supersolution 
property of u being very alike. 



Firstly, the proofs shows that (4.1) is stable, monotone and consistent. Observe 



that S is proportional to — $ in the terms outside the Hamiltonian. Since fluxes 



H are monotone by hypothesis {HI) (see 19 for details in monotone Hamiltonian 
fluxes) whenever the CFL condition (4.3) is satisfied, the monotonicity of S follows. 
The stability is ensured by the boundedness of $ and hypothesis {HQ). Finally, 
hypothesis {H8) and lemma 3.2 are used in a straightforward fashion to obtain the 
consistency properties below: 

limsup S h {x',tf,p',s',\;$>) < F*{x, q,p, s, V$) 

(x ,q ,p ,s )— »(:c,<j,p,s) 

liminf ^(a:', q',p', s', A; $) > F*{x, q,p, s, V$) 

(x' 7 q\p' ,s') — >(x 7 q,p,s) 
h^Q 

Now, chooser <G C£°(f2) such that w— $ has a strict local maximum at {xq, qo,Po, Sq) £ 
SI (without loss of generality assume {u — $>){xq, qa,pa, sq) = 0). 

First, suppose po > 0. Then there exists a ball centered in {xq, (^Po, s o) of radius 
r > such that u{x,q,p,s) < ${x,q,p,s), V{x,q,p,s) G B{{x , g ,Po, s ), r) C 
Construct sequences {x e ,q e ,p e ,s e ) — > {xo, qa,pa, sq) and h e — > as e -> such that 
Uh € {x e ,q e ,p e ,s e ) — > u{xq, qo,Po, sq) and {x e , q e ,Pe, s e ) is a maximum of Uh t — <& in 
B{{xo,qo,po, so),r). Denote Ce = { u h e — ®){x e , qe,Pe, s e ). (Remark that Q — > as 
€ 0). 

Then, u/ le < $+Ce inside the ball and since S{x e , qe,p e , s e , Uh e {x e , qe,p £ , s c ); u/ lc ) — 
0, by the monotonicity property one obtains 

S{x e ,q e ,p e ,s e ,${x e ,q e ,p e ,s e ) + Ce5 $ + Ce) < 0. 

Taking the limit (inf) e — > together with the consistency of the scheme, one obtains 
the desired inequality 

j;(2;o,<7o,Po,so,$,V$) < 0. 



Suppose now that po — 0. Construct sequences {x e , q e ,Po, s e ) — > {xq, qo,po, So) 
and h e — > as e — > such that Uh c {x e , q e ,Poi s e ) — ¥ u{xo, qo,po, sq). Then, 

lim4nf S h (x £ , q t , Po , s c , $ {x e , q t , p , s e ) , $) = liminf ($ - M$) (x e , g e , p , s £ ) 

s e ->s s e -M 
/i.-J-O /i,-vO 



($ - {M<f>)*){x ,qo,Po,s ). 



Since each ut e is a solution of (4.1 ), using lemma 3.2 the above expression yields at 
the point (x ,<jo,Po, sq): 

= ($- {M$)*){x ,q 0) p 0) s ) 

> {$-{M$)*){x ,q ,p ,s ) 

> {$-{M + <!>)){x ,q ,p ,s ) 
= J"*(xo,9o,Po,so,*,V$) 
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achieving the desired inequality. □ 

4.2. Numerical Simulations. For the numerical simulations, the numerical 



Hamiltonian H is discretized using a monotone Local Lax-Friedrichs scheme 19 
(where the two components of the gradient are explicit) : 

i +- + -\ ( a + + a~ 
ri \t, x, q; a , a ,b^,b ) = H [t,x,q; , 



2 

a + — a 



2 

where a ± = Dfv, 6 ± = D^v and the constants c a , Cb are defined 



Cb 



as 



c a = sup \d ra H(t,x,q,r)\ (4.4) 

t,x,q,r 

c b = sup \d rb H(t,x,q,r)\. (4.5) 

Setting u* = [«v? fc> *t£ ±fc> ■ • • , ^ i±fe , ««» .J, the equation 

S h (xi,q,p k ,s n+U q v n + 1 ;u*) =0 
allows an explicit expression of v n+1 as a function of past values v n : 



{( i v n _ 971™ \ 
m V - At Ik Ap !*=± + H(t n , xj, q, D~ £>+ ^? fe ) 
if (I,q,k,n) e 
min lueVV(:rj . (?) . fe ,> 5/A p 3(w,q) v n k/ if fc = 0. 

(4.6) 

In order to illustrate the work presented, a simple vehicle model is used in the 
simulations. This model allows an analytic evaluation of it's autonomy and is suitable 
for an a posteriori verification of the results. 

The switch dynamics is given by 

g(w,q) = \q-w\. 

The energetic dynamical model is given by f(u,q) = (—a x + qu, —q(a y + u)), where 
a x ,a y > are constant depletion rates of the battery's electric energy and the 
reservoir's fuel (whenever the RE is on), respectively. The control domain is taken 

U = [0,M max ]. 

Considering this dynamics, an exact autonomy of the system can be evaluated 
analytically. Given initial conditions (xo,yo) the shortest time to empty the fuel 
reservoir is given by t* — yo/{a y + u max ). The SOC evaluated at this instant is given 
by x(t*) = x(0) — t*(a x — u max ). If x(t*) < 0, it means the fuel cannot be consumed 
fast enough before the battery is depleted. This condition can be expressed in terms 
of the parameters of the model as x^{a y + u max ) < yo{a x — u max ). In this case, 
the autonomy is given by T° = xq/(cl x — it max )- If not, the autonomy is given by 
T 1 = (x Q + u max f )/a z . 

Simulations are running using Ax = Ay — 0.025, Ap — 0.05 and At is calculated 
using ( |4.3| . Several instances of xo,yo, a x , a y , it max are tested, all with a lag of 5 = 2, 
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Table 4.1 
Convergence results and running times. 



(xo,yo) 


Ax 


e 


CPU 1 running time(s) 




0.05 


0.877 


34 


(0.5,0.5) 


0.04 
0.03 


0.613 
0.860 


57 
111 




0.02 


0.326 


326 




0.05 


1.923 


28 


(0.3,0.8) 


0.04 


1.179 


46 


0.03 


0.578 


89 




0.02 


0.038 


253 



and the theoretical and evaluated autonomy are compared. For numerical purposes, 
the set of initial energies is a ball of radius 2Ax around (xo,yo) and the admissible 
region is set as K = [0, l] 2 . Remark that ( |2.14| ), ( |2.16[ ) give a natural value for 
the numerical boundary outside set K . 

The simulated instances use a x = 0.1, a y = 0.15, u max = 0.07. Tests are made 
using three initial conditions. Table 4.1 groups the error e = \T* — s*\ between 



exact autonomy and the autonomy (2.13) evaluated using the scheme (4.6| and the 
algorithm running times. 



Figures |4.1[ |4.2| and |4.3| show the minimum of the value function for all values of 
p 6 P# for q = 0, 1 and the corresponding reachable set of the instance (xo>2/o) = 
(0.3, 0.8) at times s = 2.65, s = 3.00, s — 3.75 respectively. 




Figure 4.1. Reachable set and value functions at s = 2.65. 



^ntel Xeon E5504 @ 2 x 2.00GHz, 2.99Gb RAM. 
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Figure 4.2. Reachable set and value functions at s = 3.00. 




Figure 4.3. Reachable set and value functions at s = 3.75. 
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